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1 Introduction 


This study of “Improving Pedestrian Infrastructure Inventory in Massachusetts Using Mobile 
LiDAR” was undertaken as part of the Massachusetts Department of Transportation 
(MassDOT) Research Program. This program is funded with Federal Highway Administration 
(FHWA) State Planning and Research (SPR) funds. Through this program, applied research is 
conducted on topics of importance to the Commonwealth of Massachusetts transportation 
agencies. 


1.1 Background 

Pedestrian infrastructure is one of the most vital transportation infrastructures for pedestrians 
and wheelchair users who rely on quality sidewalks to facilitate safe and uninterrupted trips in 
their everyday lives. To meet the obligations of the Americans with Disabilities Act (ADA) 
Transition Plan and to make informed investment decisions in transportation asset 
management, MassDOT is responsible for the timely identification and maintenance of 
inadequate sidewalks in its jurisdiction. It is actively seeking to improve data on existing 
pedestrian infrastructure to more clearly understand the needs for maintenance and 
construction of pedestrian facilities. It is MassDOT’s policy to provide equitable 
accommodation for all modes of transportation that seek conveyance within the 
Commonwealth. MassDOT’s consistent annual investment in sidewalks and pedestrian “curb 
cuts” is a clear example of their policies in action. 


Another major MassDOT strategic goal is to ensure that investment strategies are founded 
upon clear performance outcomes. For this purpose, MassDOT maintains high-quality data on 
bridges and pavement surfaces. This data is used to manage current conditions, develop future 
performance targets, and support investment models for the preservation and improvement of 
current and future asset performance. Given their importance to the public, bridges, and roads 
are valuable transportation infrastructure assets, and considerable annual investments are made 
to support inspection, processing, and analysis of information about them. 


Dedicated resources responsible for pedestrian infrastructure inventory and condition 
assessment do not currently exist at MassDOT. As a result, current infrastructure data is not of 
a quality or level of detail from which the organization could make informed investment 
decisions. A current sidewalk inventory dataset resides within the geographic information 
system (GIS)-based Road Inventory File (RIF), which is managed by the MassDOT Office of 
Transportation Planning (OTP). However, there are two major drawbacks of the current 
dataset due to the labor-intensive and cost-prohibitive manual data collection process in the 


current practice: the time-intensive nature of updates and the lack of sidewalk condition 
information . 


With the recent advancement of remote sensing technologies, many mobile systems with high 
data acquisition frequency and measurement accuracy, including mobile light detection and 
ranging (LiDAR), have become cost-friendly, and commercially available. The data derived 
from these mobile LiDAR systems had the potential to address the challenges mentioned above 
and deliver a comprehensive pedestrian infrastructure inventory database accurately and 
efficiently. Therefore, while there is an ongoing need for MassDOT to explore and implement 
a cost-effective, efficient, and reliable means to inventory its pedestrian infrastructure, there 
also is an emerging need for MassDOT to leverage the advancement of the new technologies 
and the availability of the emerging data to address the former. 


1.2 Objectives and Detailed Work Tasks 


This research project seeks to demonstrate the feasibility of mobile LiDAR as a viable 
technology to support efficient inventory update and condition assessment of the pedestrian 
infrastructure at MassDOT. With embedded geolocation and geometric measurement 
information, the derived database from the mobile LiDAR can be seamlessly integrated with 
the existing RIF managed by MassDOT. It is expected that the pedestrian infrastructure data 
will provide the Highway Division with accurate information from which to prioritize sidewalk 
maintenance needs. Focusing on the State Route 9 corridor, the objective of this research 
project is to collect and process data with a mobile LiDAR system, to verify and update the 
existing MassDOT’s sidewalk inventory data, and to incorporate condition information into 
the inventory geodatabase. The detailed work tasks are listed as follows: 


e Task 1 - Review of Pedestrian Infrastructure Inventory Efforts: The research team 
conducted a detailed literature review on available and ongoing research and 


implementation efforts for pedestrian infrastructure inventory and condition evaluation that 
have been made by MassDOT, other transportation agencies, and the research community. 
The research team also worked with OTP to obtain the available pedestrian infrastructure 
data that is currently residing within the GIS-based RIF. 

e Task 2—Mobile LiDAR Data Acquisition: The research team conducted a comprehensive 
data acquisition task using the mobile LiDAR sensor (i.e., Riegl VMZ-2000) along the 
State Route 9, covering 271.76 miles in both bounds. Three full data acquisition sessions 
were included to ensure the data coverage and repeatability, while two additional testing 
data acquisition sessions were included for determining the optimal sensor configurations. 


e Task 3 — LiDAR Data Processing for Pedestrian Infrastructure Inventory/Update: The 
research team developed computer-aided LiDAR processing algorithms and interactive 


point cloud processing tools for identifying the locations and geometries of the critical 
pedestrian infrastructures along State Route 9 in both bounds. Complete sidewalk 
inventory results were derived and then compared with the existing sidewalk inventory 
database provided by MassDOT. The research team evaluated the completeness and 
accuracy of the existing inventory and then updated the current pedestrian infrastructure 
inventory database with the new LiDAR-based results. As the current sidewalk inventory 
database does not contain entries for curb ramp inventory, the research team created a new 
geodatabase for referencing these pedestrian infrastructures separately. The research team 
evaluated the overall processing time for completing the inventory update using mobile 
LiDAR data. 

e Task 4—LiDAR Data Processing for Pedestrian Infrastructure Condition Extraction: The 
research team developed computer-aided LiDAR processing algorithms and interactive 
point cloud data processing tools for extracting the condition information that corresponds 
to the inventoried pedestrian infrastructure on State Route 9, including cross slope and 
grade for sidewalks and the running ramp slope for curb ramps. The research team also 
evaluated the overall processing time for completing the condition extraction using mobile 
LiDAR data. 

e Task 5 — New Pedestrian Infrastructure Geodatabase/Schema Development: The research 
team worked with MassDOT OTP to develop a new pedestrian infrastructure inventory 
geodatabase, based on the existing PedFclty database, that can be integrated with the 
current GIS-based RIF and can host both the essential inventory information and the newly 
identified condition information. The derived results from State Route 9 were populated 
into the new geodatabase as a demonstration. 


e Task 6— Reporting of Results: The final report consists of asummary of all research efforts, 
including suggestions for a successful implementation of a network-level sidewalk 
inventory and condition evaluation using mobile LiDAR data. 


41,3 Organization of this Report 

This report is organized as follows. Chapter 1 introduces the background, research needs, 
objectives, and the detailed work tasks of this research project. Chapter 2 presents the proposed 
method, including the literature review, the developed algorithms for processing mobile 
LiDAR data, and the interactive tool. Chapter 3 presents the results of the proposed method. 
Chapter 4 summarizes the findings and results of this project and recommendations for future 
studies. 
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2 Research Methodology 


The research methodology for this study consisted of three main parts: a review of existing 
data and technologies, collection of the mobile LiDAR data, and the processing of the mobile 
LiDAR data for pedestrian infrastructure inventory. Section 2.1 presents a review of the 
literature related to the existing effort for sidewalk inventory and the existing sidewalk data at 
MassDOT. Section 2.2 presents an overview of the research methodology, followed by 
Sections 2.3 through 2.6 that describe the methods for mobile LiDAR data acquisition and 
processing for generating pedestrian infrastructure information, including sidewalk inventory 
and measurements as well as curb ramp inventory and measurements. 


2.1 Literature Review 


2.1.1 Existing Efforts of Sidewalk Inventory 


In 1990, ADA developed a set of standards and guidelines for implementing environmental 
facilitators with the intention of enabling the accessibility of “the public street to people with 
disabilities with a continuous, unobstructed pedestrian circulation network to the maximum 
extent feasible.” The Americans with Disabilities Act Application Guideline (ADAAG) 
specifies a series of critical features for designing and constructing sidewalks that allow 
uninterrupted and safe trips for wheelchair users. State and local transportation agencies are 
required to assess the infrastructure’s regulatory compliance with ADA and are responsible for 
the timely maintenance of any inadequate sidewalks under their care. Although it is usually 
impractical for transportation agencies to comprehensively carry out these activities promptly 
due to the labor-intensive and cost-prohibitive nature of the manual data collection process 
involved with current practices, many of these transportation agencies have attempted to 
develop sidewalk and curb ramp inventories using both manual and automatic methods. 


The research team has identified a total of 17 transportation agencies and communities that 
have successfully implemented their sidewalk inventories, including one state agency, the state 
of New Jersey (New Jersey Department of Transportation, 2007); three county/regional 
agencies, the counties of Franklin, Delaware, Fairfield and Licking in Ohio (Ohio Department 
of Transportation, 2018), the County of Delaware, Pennsylvania (Planning Department, 2018) 
and the County of Boone, Missouri (MID-MO Regional Planning Commission, 2014); and 
thirteen city agencies: the City of Tucson, Arizona (Cole and Leon, 2012), the City of Bellevue, 
Washington (Loewenherz, 2010), the City of Urbana-Champaign, Illinois (Department of 
Transportation, 2016), the City of Boston, Massachusetts (Boston Public Works Department, 
2015), the City of Toronto, Canada (City of Toronto, 2015), the City of Lee’s Summit, 
Missouri (Burns & McDonnell Engineering Company, 2009), the City of Texarkana, Texas 
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(Data Transfer Solutions, LLC, 2017), the City of Ironton, Ohio (Mannik & Smith Group, Inc, 
2018), the City of Columbia, Missouri (Public Works Department, 2018), the City of 
Lakewood, Colorado (Hometown Colorado Initiative, 2015), the City of Los Angeles, 
California (Gooch, 2017), the City of San Diego, California (Street Division, City of San 
Diego, 2015), and the City of Rutland, Vermont (Department of Public Works, 2013). 


Through the review of these implemented inventories, the following findings were identified 
from the perspectives of the inventory coverage, the employed technology, the information 
included in the inventory, and the cost and productivity of the projects. 


Coverage: The effort on inventorying pedestrian infrastructures has been a continuously 
ongoing process by public agencies and communities. Some of the earliest efforts can be 
traced back to more than a decade, such as at the New Jersey Department of Transportation 
(NJDOT) (New Jersey Department of Transportation, 2007) and in Bellevue, Washington 
(Loewenherz 2010), but most of the inventories have only concluded recently. However, 
only NJDOT, out of the 17 implemented inventories, covers an extensive length of the 
network on a state level, while the rest of the inventories only covered a very small region 
and network within the city’s or county’s jurisdictions. While the financial burden partially 
constrains a broader implementation, the limitation of the existing technologies plays a 
more significant role. 

Technology: The advancement of inventory technologies has played an essential role in 
successful implementations. From a data acquisition perspective, GPS- and camera- 
equipped handheld computers have been widely utilized to inventory the existing sidewalk 
infrastructure. To improve the productivity of the data acquisition, integrated mobile 
systems, e.g., Ultra-Light Inertial Profiler (ULIP), were introduced (Loewenherz, 2010), 
and inventory efforts were conducted with the recurrent highway pavement performance 
survey (Data Transfer Solutions, LLC, 2017; New Jersey Department of Transportation, 
2007). However, most data acquisition efforts are conducted manually, e.g., a walking 
survey. From a data processing and integration perspective, GIS technologies have been 
broadly utilized, so sidewalk inventories can be conveniently integrated and easily shared 
with the public, such as in the City of Toronto (City of Toronto, 2015) and the City of San 
Diego (Street Division, City of San Diego, 2015). However, most of the current data 
processing efforts remain manual, especially when it comes to detailed condition properties 
of sidewalks or ramps (Loewenherz 2010). Other applications of emerging technologies, 
e.g., aerial orthophotos (Department of Public Works, 2013), were developed with 
promising results that led to successful implementation. However, none of the 
transportation agencies or communities have attempted the LiDAR-based method. 
Inventoried Information: A complete sidewalk inventory covers the primary pedestrian 
infrastructure of sidewalks and curb ramps with both their locations and conditions (Ai and 
Tsai, 2016b). However, most of the implemented inventories focus primarily on the 
locations of the sidewalks. Only a few inventories contain the locations of curb ramps or 
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the conditions of the sidewalks and curb ramps. The inventories in Tucson, Arizona (Cole 
and Leon, 2012), Bellevue, Washington (Loewenherz 2010), and Urbana-Champaign, 
Illinois (Department of Transportation 2016) contain the most comprehensive information. 
It should be noted that due to the limitation of the technologies employed in the data 
acquisition and processing, and the constraints of project budgets, these inventories were 
conducted in multiple stages. The conditions and locations for the sidewalks were 
progressively implemented, as were the locations and conditions of the curb ramps. 

e Cost and Productivity: The research team identified most of the projects were concluded 
within two years, thanks to the smaller coverage of the inventoried network. However, due 
to the different levels of details in various projects, the labor hours for different 
implementations varied significantly. The research team only identified limited 
information about the cost of these successful inventories. Despite the employed 
technologies in these implementations, the project costs tended to increase significantly if 
additional information was included. For example, the City of Bellevue, Washington, and 
Lee’s Summit, Missouri inventoried similar network lengths of sidewalks, but the City of 
Bellevue spent over $100,000 more (more than 30% of the total cost) than the City of 
Lee’s Summit for an additional 4,586 curb ramp locations (Burns & McDonnell 
Engineering Company, 2009). 


2.1.2 Technologies and Methods for Sidewalk Inventory 


While many sidewalk inventories have been successfully implemented by local and state 
transportation agencies, the research team identified the implementation of the coverage, the 
accuracy, the comprehensiveness of the inventoried information, and the productivity of these 
inventories were constrained by the limited applications of the emerging technologies and the 
lack of automated or semi-automated methods. With the recent advancement of remote sensing 
technologies, many mobile systems with high data acquisition frequency and measurement 
accuracy (e.g., video log imagery, mobile LiDAR) have become cost-friendly and 
commercially available—with better data quality. In recent years, there have been many 
existing studies on developing automated or semi-automated sidewalk extraction and 
measurement methods by taking advantage of these emerging remote sensing technologies. 


The early studies mainly focused on images coupled with GPS positioning mechanisms. Most 
of these studies dealt with the localization of pedestrian infrastructures. A broad spectrum of 
imagery was attempted to balance the coverage of the network with the granularity of the 
sidewalk data, including aerial imagery, video log imagery, crowdsourcing imagery, etc. Guo 
et al. (2004) segmented the road areas in aerial imagery by matching digital line graph (DLG) 
maps, and sidewalk edge detection was subsequently employed based on the preliminary 
segmented results. A few more recent studies continue with a similar pipeline by Guo et al. 
(2010). However, due to the limited resolution of the imagery, it was less feasible to extract 
the curb ramp conditions and locations using an image-based method. Hara et al. (2013, 2014) 
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developed a support vector machine (SVM)-based computer vision technology to identify the 
presence of curb ramps using Google Street viewer images. The histogram of the Gaussian 
feature was employed as the primary feature associated with curve ramps. A semi-automated 
interface was also developed to facilitate the manual process in case the automated algorithm 
failed. Ai and Tsai (2016b) developed a deformable part model (DPM) that used high- 
resolution video log images to identify the accuracy boundary of the curb ramps. Although the 
existing image-based sidewalk and curb ramp extraction algorithms have shown promising 
results, the major drawbacks of the image-based method (e.g., a lack of accurate geometry 
measure) prevent these methods from evaluating the geometry of the sidewalk and curb ramps. 


With the emergence of new connected and autonomous vehicles, mobile LiDAR data, and 
other depth-enabled data have become widely available. Most of the recent studies have 
focused on LiDAR-based methods. These methods efficiently identify the locations of 
sidewalks and curb ramps. They also accurately evaluate the geometry and condition of these 
infrastructures. However, limited numbers of studies have focused on sidewalk extraction or 
inventory. In this technical memo, the research team further investigated three studies for 
pedestrian infrastructure identification and evaluation in detail to assess the state-of-art 
performance of these methods, and to reveal the potential applications of LiDAR-based 
methods. Each of these three methods is representative of the state-of-art LiDAR processing 
techniques, namely the geometry-based method, the reflectance-based method, and the scan- 
based method, which have shown promising results for sidewalk and roadway extraction. 


Geometry-based Method 


As sidewalks and curb ramps are distinguishable by their unique geometrical shapes (e.g., edge 
or plane) and geometrical measurements (e.g., length, width, adjacency), the geometry-based 
segmentation method has been identified as an efficient and accurate approach. Balado et al. 
(2018) proposed such a new approach to automatically detect and classify five categories of 
urban ground elements, including roads, sidewalks, treads, risers, and curbs. There are mainly 
two steps in the methodology, as shown in Figure 1. 


e Step 1 focuses on a planar segmentation and refinement to divide point clouds into planar 
regions. After the initial normal-based segmentation, split and merge operations are 
followed to reduce segmentation errors caused by the quality of the input data or the 
disturbance caused by small road objects. Finally, the coplanar refinement and road- 
sidewalk refinement are applied to fine-tune the identified object based on the coplanar 
constraint and the normal variance constraint, respectively. 

e Step 2 focuses on classifying the planar region into different ground element categories 
based on empirical geometry constraints. In this classification, four classes (risers and 
curbs, treads, sidewalks, and high vertical elements) are considered for classifying point 


cloud regions according to the geometry constraints, including the horizontal and vertical 
dimensions and the tilting angles. 
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Figure 1 Flowcharts for Steps 1 and 2 of the sidewalk extraction algorithm (Balado et 
al., 2018) 


The final topologic validation step is applied to finalize the process based on the empirical 
ground object adjacency. All the classification results and their spatial relations are stored in a 
graph library that is scalable to larger datasets. The methodology was tested in four real-world 
datasets, and the results show a correct classification rate of 97.0%. However, researchers 
noticed that the proposed method shows better results for larger elements (e.g., sidewalks, 
roads, etc.), while underperforms for smaller elements (e.g., curbs, treads, risers, etc.). This is 
primarily attributed to the more distinguishable coplanar geometry in larger elements, and, in 
addition, attributed to the more points associated with, the larger elements that reveal more 
distinguishable geometry constraints. 


Reflectance-based Method 


Besides the unique geometrical shape and adjacent constraints, many of the road objects, e.g., 
pavement marking (Soilan et al., 2018) and signage (Ai and Tsai, 2016a), are also 
distinguishable because of their unique features captured by LiDAR, such as reflectance 
intensity. Soilan et al. (2018) proposed a method for the detection of road and sidewalk 
networks by taking advantage of the unique reflectance feature. The urban ground-level, 
semantic information (e.g., road edges, sidewalks) can be identified efficiently through the 
point cloud preprocessing. Figure 2 shows the overall flow of the proposed algorithm, 
consisting of five primary steps, including point cloud preprocessing, elevation-based 
segmentation, ground extraction, curb map definition, and roadway feature extraction. 
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Figure 2 Flowchart of the proposed road feature extraction algorithm (Soilan et al., 
2018) 


Step 1 focuses on the preprocessing of the point cloud by normalizing the reflectance 
intensity based on the laser scanning model, and by creating a saliency map derived through 
a distance projection. The normalization process takes advantage of the unique physical 
characteristics of the point cloud reflectance, determined by beam distance and incidence 
angle, and efficiently homogenizes the points for all road objects. The saliency mapping 
process is then applied to identify the point clusters whose surfaces only change gradually, 
such as roofs, balconies, and ground segments, and the point clusters whose surfaces 
change drastically—fag¢ades, poles, lateral sides of vehicles, etc. 

Steps 2-5 follow a similar approach as other geometry-based methods by taking advantage 
of the geometrical shapes (e.g., edges and planes) and geometrical measurements(e.g., 
elevations and elevation gradients). It should be noted that although features like 
reflectance can be used to efficiently extract many road objects that are effective visual 
cues and spatial references for sidewalks, geometrical shapes and measurements remain 
critical for extracting sidewalks and other pedestrian infrastructures. Therefore, geometry- 
and reflection-based methods are usually coupled to produce a better performance 
regarding accuracy and productivity. 


Besides sidewalks, the proposed method is also applied to extract pavement and pavement 
markings. The results show promising results of F-scores around 95% for pavement as well as 
sidewalks, determined through a test conducted over a high-density urban LiDAR data set for 
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a portion of Dublin, Ireland. However, the computation efficiency remains a challenge due to 
the employed complex fagades extraction and ground region-growing methods. 


Scan-based Method 


While most of the existing LiDAR-based methods focus on the combination of geometry and 
reflectance features for more accurate and efficient road object extraction, one of the critical 
features is often overlooked: the scanning pattern. If this piece of crucial information is known 
beforehand, which is usually the case, accuracy and productivity can be significantly improved. 
Ai and Tsai (2016b) proposed a method for sidewalk extraction by taking advantage of the 
linear scanning pattern of the collected data using a line-scanning LiDAR. Instead of treating 
the LiDAR point cloud as an arbitrary cluster, the proposed method treated the linear scanning 
pattern as a continuous sequence with linkage information. Therefore, instead of a 3-D 
segmenting for all the points, a much simpler 2-D segmentation for each scan that is 
perpendicular to the traveling direction is applied. Figure 3 shows the scan pattern and the 
detail of the segmentation of a scan line. 


Figure 3 Illustration of the 2-D LiDAR scan-based segmentation algorithm (Ai and 
Tsai, 2016b) 


Although a 2-D segmentation works effectively with better efficiency than a 3-D segmentation, 
a strong assumption that an ideal roadway cross-section comprised of four zones, including 
driveway zone (gray), planter zone (green), pedestrian zone (blue), and frontage zone (yellow) 
must be made when using this approach. Such an assumption does not often hold due to the 
presence of a parked vehicle, interruption of the sidewalk (such as an intersection), or other 
interfering objects (e.g., tree trunk, fire hydrant). Therefore, a segmentation connection method 
can be introduced to adjust the segmentation results and to link results from consecutive scans 
into a continuous boundary. Figure 4(a) illustrates the original segmentation results in 
successive scans, where the dash lines represent the laser scans, and red dots represent the raw 
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segmentation results. The segmentation results are clustered longitudinally based on the k- 
nearest neighbor (k-NN) method. Figure 4(b) illustrates the clustered results, where different 
circles represent the label of the clusters. The spatial relationship in the longitudinal direction 
is then measured to further merge the clusters that are along the same trace in the longitudinal 
direction, as shown in Figure 4(c). The spatial relationship in the transverse direction is 
measured to remove the collinear segments with a shorter length. Figure 4(d) illustrates the 
results after the removal of the red cluster. A B-spline algorithm is then introduced to connect 
different segmentation points smoothly. Figure 4(e) shows the results of the final segmentation 
results, where different colors represent the corresponding zones of the driveway (gray), 
planter (green), a pedestrian (blue), and frontage (yellow). 
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Figure 4 The process of the cross-section segmentation connection method (Ai and Tsai, 
2016b) 


The results show that out of the two-mile tested sections, more than 98.3% of sidewalks can 
be successfully identified with minimal false detection (defined as excessive obstructions by 
utility facilities and shuttle buses). More importantly, the processing time for extracting the 
sidewalk using the 2-D method is significantly faster than the traditional 3-D segmentation 
method using both geometry and reflectance features. Beside sidewalk extraction, the proposed 
study would also develop a data fusion technique for integrating the video log images with the 
mobile LiDAR data. The image-based method using DPM for curb ramp identification can 
effectively project its results to the LiDAR point cloud; therefore, the corresponding slope 
measurements can be conducted subsequently for ADA compliance. 
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Summary 


Although LiDAR-based methods are applied in many other road object extractions and show 
promise in sidewalk inventory, only limited numbers of such studies have been identified 
through this technical review. The existing methods using geometry, reflectance reference, and 
scanning patterns have been reliable in extracting the locations of sidewalks, and some of them 
have proven to be efficient in their use of time and resources. However, there remains room 
for performance improvements for these methods in regard to both accuracy and productivity. 


e For the scope of sidewalk extraction, while Ai and Tsai (2016b) have developed a method 
with considerable accuracy and efficiency, their method remains to be validated with a 
larger dataset, and the strong assumption of the scanning pattern needs to be improved for 
a more practical application. Therefore, the research team carried out the work task of 
sidewalk extraction based on the method proposed by Ai and Tsai and made critical 
improvements to address the technical challenges that may hinder this method from large- 
scale implementation, including 1) it relies on strong priors, e.g., sensor configuration and 
scene context to tune the parameters for consistent results; 2) it can only handle simple 
scenes where features of interest are significant; 3) it relies on an iterative and sequential 
search and match strategy for sidewalk extraction that can be time-consuming and 
infeasible for large-scale data sets. 

e For the scope of curb ramp extraction and condition evaluation, while a few studies have 
focused on curb ramp identification using image processing and crowdsourcing (Hara et 
al., 2013 and 2014), the research team only identified Ai and Tsai’s method (2016b) as 
being capable of both extracting the curb ramp locations and measuring their critical slope 
information. Therefore, the research team carried out the work task of curb ramp extraction 
by directly adopting the method proposed by Ai and Tsai. 


2.1.3 Current Efforts of Sidewalk Inventory in MassDOT 


MassDOT actively collects roadway inventory data and maintains a centralized geodatabase 
(the RIF) to facilitate data consolidation, data processing, data visualization, and data sharing. 
The existing sidewalk inventory data resides in the RIF, including the location of the sidewalk, 
curb type, left and right sidewalk width, pedestrian surface type, and pedestrian facility type. 
MassDOT has also developed and maintained a separate geodatabase called CurbRamp to 
facilitate the data integration that is dedicated for curb ramp facilities. The existing curb ramp 
inventory data include the curb ramp location, inspection results (missing components), 
inspection comments, current status, visual deficiency, and other factors. Figure 5 shows a 
screenshot of the RIF data fields with the highlight of the corresponding sidewalk infrastructure 
information (in brown) and the CurbRamp data fields (in blue). 


13 


RouteName SRI EB 
Townld 340 


IsMissingR 

isMissingS 

IsMissingC 

Insp. Notes flat conc walk 
Districtid 1 


Number of Travel Lanes 2 
Opposite Number of Travel Lanes 0 
Surface Type 6 
Surface Width 2 
Shoulder Width Right 2 
Shidr_RUT ed LegaciD 16138 
Shoulder Width Len 0 ashe Ramoied 
Shice_14 7 o Dstret Rank <Null> 
Shoulder Width Ler 2 
Shidr_UL_T 3 
Median Width 0 

0 

3 

4 


deficient False 
Reinspectn <Null> 
AssetiD <Null> 
Program Notes <Null> 
x -809327 1.745837 
5217265.103189 


y 


Toss Slope < 
Off-road Path Width <N 
Street Operation 2 
‘Speed Limit 35 
Speed Limit Opposing Direction 35 
Speed Limit 35 
Truck <Null> 
Truck Exclusion Type 1 
Truck Exclusion Time 1 
Truck Permit <Null> 
Truck Network <Null> 
Truck Route 2 
Right Of Way Width 42 
‘Structural Condition 1 
Terrain Type 1 . 
Toll Road <Null> 
Metropolitian Highway System <Null> 


lall> 
jutl> 


Figure 5 A screenshot of the RIF data field (in brown) and the Curb Ramp data fields 
(in blue) 


MassDOT has made an effort to inventory the sidewalks and curb ramps in its network and has 
developed a comprehensive data structure to house and analyze the information (the 
geodatabases of RIF and CurbRamp). However, the following limitations have been identified 
by the research team: 


e Sidewalk location: The current sidewalk inventory is conducted based on the Road 
Segment ID stored in the RIF. While it is convenient to maintain and reference the existing 
highway performance management system (HPMS) and the pavement management 
system, the delineation of the sidewalks may not align well with the delineation of the Road 
Segment. 

e Sidewalk condition: While the data fields for sidewalk conditions include the sidewalk 
width measurements, other important geometries (including sidewalk grade, sidewalk 
cross slope, etc.) have not been included. Also, as the sidewalk widths were estimated 
based on the manual inspection and the orthograph imagery analysis, the accuracy remains 
to be validated. 

e Curb ramp location: The current curb ramp inventory has been spatially referenced based 
on the curb ramps’ XY coordinates. While the relative coordinates concerning the road 
inventory are all populated in the map (as shown in Figure 5), the locations and existence 
of these inventory ramps remain to be validated. 
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e Curb ramp condition: While the data fields for curb ramp condition include inspection 
results conducted by the field engineers, they only include whether the curb ramp misses 
any component, and whether the curb ramp is currently active or not. Some of the critical 
geometrical information is missing, such as the curb ramp slope. 


Focusing on State Route 9 in this research project and based on the collection of the existing 
pedestrian infrastructure data from MassDOT, the research team aims at addressing the 
limitations mentioned above using the LiDAR-based method. It is expected that the LiDAR- 
based method will be able to accurately validate the existing inventory locations for the 
sidewalks and curb ramps and to populate the corresponding condition information efficiently. 
Figure 6 shows the overview of the scoped region with the highlighted road sections from the 
RIF geodatabase (in red) and the inventory curb ramps from the CurbRamp geodatabase (in 
green). 


Figure 6 Illustration of the SR9 corridor of interest for this research project 


2.2 Methodology Overview 

In this study, the research developed a complete data processing methodology for sidewalk 
inventory and curb ramp condition evaluation from the raw LiDAR data acquisition to GIS 
integration. This methodology consists of six key steps, including Data Acquisition, Point 
Cloud Segmentation, Sidewalk Extraction, Curb Ramp Extraction, Key Feature Measurement, 
and the final Sidewalk Inventory. Figure 7 shows an overview of the research methodology. 
The subsequent sections present the detailed methods and algorithms developed in this study 
for each key step of this complete data processing methodology. 
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Figure 7 Overview of the research methodology 


The data acquisition system used in this study is an integrated mobile LiDAR system, RIEGL 
VMZ-2000, that consists of three primary components: the LiDAR sensor, the precise 
positioning system, and the camera system. The LiDAR sensor is used to acquire the point 
cloud of the roadway, including pedestrian infrastructures. Each point consists of the precise 
position information that is derived from the integrated precise positioning system. The 
integrated precise positioning system is used to acquire accurate coordinates that are composed 
of a global positioning system (GPS) and an inertial measurement unit (IMU). The camera 
system is used to capture video log images that are registered to the LiDAR sensor. In this 
study, the point cloud data acquired by the LiDAR sensor was used for the subsequent 
processing, while the precise positioning system and the camera system were used for 
positional correction and visual reference for the point cloud, respectively. 


The current LiDAR sensor can produce 400,000 measurements per second in both line- 
scanning mode and radar-scanning mode. For the application of the corridor scanning with the 
best point cloud density, the line-scanning mode was selected. Figure 8 shows the vertical and 
horizontal configurations for the line-scanning mode. 


e For the vertical configuration, the scanning line of the LiDAR sensor is aligned 
perpendicular to the ground when the vehicle makes a longitudinal motion. The scanning 
line forms a 100° vertical fan to cover the road surface, especially the roadside objects. To 
acquire the point cloud with better homogeneity of point cloud densities, the frequency of 
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the LiDAR sensor and the LiDAR heading angle were configured at 75 Hz (i.e., lines per 
second) and 135° (i.e., the angle to the vehicle driving direction), respectively. 


e For the horizontal configuration, the scanning line of the LiDAR sensor is aligned parallel 
to the ground when the vehicle makes a longitudinal motion. The scanning line forms a 
100° horizontal fan to cover the road surface at the back of the vehicle. To balance the 
density and the coverage of the point cloud , the frequency of the LiDAR sensor and the 
LiDAR pitching angle were configured at 75 Hz (i.e., lines per second) and -15° (i.e., the 
angle to the level of the vehicle), respectively. 


Figure 8 Mobile LiDAR installations for both vertical and horizontal configuration 


With different trials of configuration, the vertical configuration was selected for better point 
cloud density that is associated with the roadside features. In this study, the radar mode was 
not evaluated in detail due to the now-known fact of inhomogeneous point cloud density, 
because the radar mode is primarily designed for stop-and-go operations. 


2.4 Point Cloud Segmentation 

The objective of point cloud segmentation is to decompose the large-scale, raw point cloud 
data into feature-rich, meaningful groups so that the subsequent sidewalk extraction algorithms 
can be applied efficiently. Previous research has been done for segmenting LiDAR point cloud 
with high accuracy using deep learning. Liu et al. (2017) introduced a deep reinforcement 
learning method to parse the large-scale 3D point clouds and map the raw data to make the 
classification. Maturana et al. (2015) used VoxNet, a supervised convolutional neural network 
(CNN) integrating a volumetric grid, to realize the robust object recognition . However, most 
neural networks do not directly consume the raw point clouds but require preparation, such as 
voxelization, which can be ineffective. PointNet (Charles et al., 2017) is a pioneering CNN 
that directly processes the raw point clouds, which reflects the permutation invariance of the 
inputting points. PointNet has increasingly been applied to object classification and semantic 
segmentation thanks to its high efficiency. There are two steps in the formulation of PointNet: 
the spatial encoding of every point is obtained first, and then all individual point features are 
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aggregated to a global point cloud signature. Although PointNet is an efficient and effective 
neural network, it cannot capture the local structures which limit PointNet from complex 
scenes. The hierarchical neural network, PointNet++ (Qi et al., 2017), as an extension of 
PointNet, was introduced to solve this problem. It applies PointNet recursively on a nested 
partitioning of the input point set. PointNet++ can achieve both the robustness under density 
variation and a high level of detail captured in the local feature. However, very few efforts 
have been made to apply PointNet++ to road scene segmentation. Therefore, this study 
introduces PointNet and PointNet++ for efficient point cloud segmentation. 


In this subsection, the algorithms for both PointNet and PointNet++ are briefly described, 
followed by the customized training process used by the research team. 


2.4.1 PointNet 


PointNet directly processes the input point clouds without transforming to other data structures, 
such as voxel or graph, and outputs each point segmentation label corresponding to the input 
per point. This network architecture maintains three properties of the input point clouds: it is 
unordered, displays interaction among points, and shows invariance under transformations. 
PointNet can be used in many tasks, such as 3D shape classification, shape part segmentation, 
and scene semantic segmentation. This component of PointNet mainly concentrates on scene 
semantic segmentation. 
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Figure 9 PointNet Architecture 


The network architecture of PointNet is shown in Figure 9. There are three key modules in 
PointNet architecture: symmetry function, local and global information aggregation structure, 
and joint alignment network. The symmetry function embedded in the max-pooling layer 
aggregates the information of the input points for unordered input. The local and global 
information aggregation structure is used for the point segmentation with high effectivity. Both 
the input points and point features are aligned by two joint alignment networks. 
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In order to make an invariant permutation for the input points, a simple symmetry function is 
used to aggregate the information of the input data. The function is defined via a point set by 
transforming elements in the set. 


ftp Xn) * GAO), An) 


Where f: 2%” > R, h:R™ > RK and g: RX x---x R¥ > R is a symmetry function with n 
RX s multiplications. h is approximated by a multi-layer perceptron network. g is 
approximated by a composition of a single variable function and a max-pooling function. 


The combination of local and global information is achieved by this structure for the point 
segmentation (as shown in Figure 9). The global features are fed back to each point feature by 
concatenating the global feature with per-point features. The new per-point features are 
extracted based on the combined point features generated in the above step. At the same time, 
the per point feature contains local and global information. 


A joint alignment network keeps the point set invariant while undergoing geomatic 
transformations. A mini-network (T-net, as shown in Figure 9), which resembles the big 
networks, is used to predict an affine transformation matrix. The network directly applies this 
transformation to the coordinates of input points. Another alignment network is also applied 
in the alignment of the feature space with the help of adding a regularization term to the 
softmax training loss. A feature transformation matrix can be predicted by this network to align 
the features from different input point clouds. 


2.4.2 PointNet++ 


The hierarchical neural network PointNet++ was introduced since PointNet cannot capture the 
local features introduced by the metric. PointNet++ is an extension of PointNet with the same 
hierarchical structure. Two issues are solved by this neural network: the need to generate the 
partitioning of the point set and to abstract the local features or point sets through a local feature 
learner. The farthest point sampling (FPS) algorithm is applied to generate the partitioning of 
a point set. PointNet++ is used to abstract the sets of the local points or features in a nested 
partitioning of the point set. 
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Figure 10 PointNet++ Architecture (Qi et al., 2017) 


PointNet++ architecture is visualized in Figure 10. There are mainly two applications: 
classification and set segmentation. The second application is discussed in this subsection as 
the relevant background. The hierarchical structure is made of several set abstraction levels. 
Each set abstraction level contains a Sampling layer, a Grouping layer, and a PointNet layer. 


The input N x (d + C) matrix from N points with d — dim coordinates and C — dimpoint 
feature is processed and an N’ x (d + C’) matrix of N’ subsampled points with d — dim 
coordinates and C’ — dim point feature summarizing local context is outputted in a set 
abstraction level. 


In the Sampling layer, a set of points defining the centroids of local regions from the input 
points are selected by using FPS, which has better coverage of the whole point set. A subset of 
points {x;,,X;i,,-.-,X;,,} 1S chosen from the input points {x,,2,...,Xn}, where x;,is the most 
distant point (in the metric distance) from the point set {%;,,Xj,,--., Xi;_4} with regard to the 


rest of the points. 


In the Grouping layer, a point set with size N x (d + C) and the coordinates of the centroids 
with size N’ x d are inputted. The local region sets are generated by finding the points in the 
neighbor area of the centroids based on the ball query, which is preferred for semantic point 
labeling. The groups of point sets with size N’' x K x (d + C) (Kis the number of points 
around the centroid points) are outputted. Each group matches a local region in this layer. 


In the PointNet layer, the size of the input points in local regions is N’ x K x (d + C). The 
local regions are abstracted by encoding local region patterns into feature vectors using a mini- 
PointNet. The size of the output points is N’ x (d + C’). 
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Two types of grouping under non-uniform sampling density when combing features in 
different scales were proposed: multi-scale grouping (MSG) and multi-resolution grouping 
(MRG). As shown in Figure 11(a), features at different scales are concatenated to form a multi- 
scale feature in MSG. MRG is introduced since MSG generates expensive computation for 
each centroid point at large scale neighborhoods. In MRG (as shown in Figure 11(b)), a region 
features at some level L; is a concatenation of two vectors. Summarizing the features at each 
subregion from the lower level L;_, obtains one MSG (left) by the set abstraction level. 
Processing the raw points in the local region obtains the other (right) by a local PointNet. The 
weights of the two vectors are different in different densities of a local region. 


concat 


concat 


Figure 11 Illustrations of MSG and MRG2 (Qi et al., 2017) 


In the set segmentation, the point features of all original points are obtained by propagating the 
features from the subsampled points to the original points. For point feature propagation, two 
measures (distance-based interpolation and across-level skip links) are used in the hierarchical 
structure (as shown in Figure 10). The inverse distance weighted average based on k nearest 
neighbors is used to interpolate feature values f. 
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The interpolated features are concatenated with skip-linked point features from a set 
abstraction level. The concatenated features then pass through a PointNet unit. Each point’s 
feature is updated by some shared fully connected and Rectified Linear Unit (ReLU) layers. 
The process is repeated until the features are propagated to the original point sets. 


2.4.3 PointNet++ Model Training 


In this study, the PointNet++ neural network is trained and validated using the semantic-8 
dataset (Hackel et al., 2017), which is popular among researchers. The semantic-8 dataset has 
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large-scale point clouds with over one billion labeled points for training and validation. The 
semantic-8 dataset with eight class labels includes 15 scenes (streets, squares, village, etc.). In 
this study, the improved six classes are used for the road scene segmentation, including man- 
made and natural terrain (MNT), vegetation including low and high vegetation (V), buildings 
and structures (BS), hardscape (HS), scanning artifacts (SA), and cars (C). The testing dataset 
was collected on State Route 9 and manually labeled as six classes according to the semantic- 
8 dataset for evaluating the trained model. The point clouds of the MNT were obtained by the 
prediction of the trained model using the unlabeled collected dataset. Figure 12 shows 
examples of the six classes defined in this study. The basic environment of the road scene 
segmentation based on PointNet++ is shown in Table 1. 
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Figure 12 Examples of the six classification classes 


Table 1 Basic environment of the road scene segmentation based on PointNet++ 


Type Description 

Processor Intel(R) Core (TM) i7-8700 CPU @ 3.20GHz 3.19GHz 
Memory (RAM) | 32 GB 

GPU NVIDIA GeForce GTX 1080 

System type Ubuntu 18.04 LTS, 64-bit Operating system 

Others TensorFlow 1.8 


The datasets “sg27_5”, “sg28 4”, “untermaederbrunnen1,” and “untermaederbrunnen3” in the 
semantic-8 dataset were used as the validation datasets, and others were used as the training 
datasets. It required 24 hours, 8 minutes, and 4 seconds to train and validate the model when 
the epochs were 500. One epoch is a pass through the entire training dataset. 


22 


Overall Accuracy (OA) and Intersection over Union (IoU) are used as the metrics. OA is an 
evaluation metric measuring the detection accuracy. IoU is the ratio between the intersection 
and the union of two sets (detection result and ground truth). It is calculated based on the 
following formula. 


DetectionResult N GroundTruth 


loU = ———___— ETE 
" DetectionResult U GroundTruth 


The detection result is correct if lJoU => 0.5. IoU will be 1 if the detection result is equal to the 
ground truth. The changing accuracy, OA, and IoU of each class per epoch are shown in Figure 
13. The training model performs at a high accuracy of 94.65% when the epochs are 500, as 
shown in Figure 13(a). Figure 13(b) depicts that the high IoU 0.53 is obtained in this trained 
model when the epochs are 500. The training model is stable based on the near-horizontal lines 
in both Figure 13(a) and (b) when the epoch arrives at 500. The changing IoU of each class per 
epoch is shown in Figure 13(c). The MNT class performs at a higher IoU of 0.94 when the 
epochs are 500. The model for training the MNT class also achieves stability with the 500" 
epoch. 


Accuracy per epoch loU per epoch 


° 100 200 300 400 500 ° 100 200 300 400 500 
Epochs Epochs 


(a) Accuracy per epoch (b) loU per epoch 
Man-made and natural terrain - Vegetation Buildings and structures 
0.0 0.0 0.0 
ft) 500 0 500 0) 500 
Hard scape Scanning artefacts Cars 
1.0 1.0 1.0 
0.0 0.0 0.0 
0) 500 0) 500 0 500 


(c) loU for each class per epoch 


Figure 13 Accuracy, IoU, and IoU of each class per epoch 
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2.4 Sidewalk Extraction 

The objective of this step for sidewalk extraction is to accurately extract the boundaries of the 
sidewalk from the segmented point cloud that is associated with the man-made and natural 
terrain. In other words, while many of the existing methods for automated sidewalk extraction 
using a LiDAR point cloud have shown promising results, the 2-D segmentation method 
developed by Ai and Tsai (2016b) has shown particularly high levels of efficiency without 
compromising accuracy. However, due to the limited features that can be used from the 2-D 
profile, the performance for the regression within each 2-D profile varies and often relies on 
the post-processing for reconnecting the segmented profile longitudinally. In this study, a 
stripe-based extraction method is proposed to overcome the challenge 2-D segmentation 
method encounters. More importantly, the elevation and lateral offset filtering processes that 
rely on the sensor configuration and the scene context can be effectively removed without 
impacting the overall performance. 


A stripe is defined as the basic processing unit for sidewalk extraction, which covers 3 ft. 
(approx. 1 meter) of distance along the traveling direction. All the points associated with the 
man-made and natural terrain class and bounded by a stripe are converted into an octree 
structure (Vo et al., 2015) and processed in twofold, including: 


e Stripe Splitting: The split process of the algorithm is to recursively split the point cloud 
until each node of the octree only contains points that satisfy the coplanar criterion. Figure 
14(a)-(c) shows an illustration of the split process using a 2-D example. Figure 14(a) shows 
space containing all point clouds within the stripe as the root node. Since the coplanar 
criterion is not satisfied, space is split into eight sub-spaces (only four shown in Figure 
14(b). The points set in nodes 1 and 2 passes the coplanar criterion, so no further split is 
required. The points set in node 0 will be further split into eight sub-spaces, as shown in 
Figure 14(c). Since the points set in all the nodes pass the coplanar criterion, no further 
split is required. 

e Stripe Merging: The merge process of the algorithm is applied to combine the neighboring 
nodes if the points in the combined node still satisfy the coplanar criterion. The merging 
process will be exhaustively conducted until no neighboring nodes can be merged without 
violating the coplanar criterion. Figure 14(d)-(f) shows an illustration of the merge 
processing. As shown in Figure 14(d), the points in neighboring nodes can share a similar 
normal direction, which indicates that these points should be merged into the same cluster. 
Therefore, for each node, the coplanar test is conducted by including the points from one 
of the neighboring nodes. If the coplanar criterion is satisfied, the two nodes are merged 
into one, as shown in Figure 14(e). The merging process is exhaustively conducted for all 
the nodes until no further merging can be conducted. Figure 14(f) shows the results of the 
clustering. Two nodes (i.e., two clusters) are identified in this point cloud. 
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Figure 14 Illustrations for the splitting and merging algorithm 


The coplanar criterion is determined using the PCA. The following equations are constructed 
for PCA computation for the optimal normal of the given data, i.e., points within a node. The 
solution is obtained from the three eigenvectors. The eigenvectors represent the three axes of 
the points, while the eigenvalues denote the square sum of points deviating along the 
corresponding axis. Therefore, the minimum eigenvalue represents the variation along the 
normal direction of the best-estimated plane using the points within each node. 


k 
1 y Pare 
C= =) —p):(pi-p)', C+¥;, =A; + vf € {0,1,2} 
i=1 


where k is the number of points in the point cloud pj, p is the centroid of the cluster, A; is the 
jth eigenvalue of the covariance matrix C and 7; is the jth eigenvector. Coplanar points should 
result in very small variation along the normal direction of the estimated plan. Therefore, the 
coplanar criterion is defined as min(A;) < A). The selection of the threshold A is determined 
by the systematic range measurement error of the LIDAR sensor. Figure 15 shows an example 
of the strip-based sidewalk extraction results. The direction of the arrows shown in Figure 15 
represents the normal direction of the plane, and the magnitude of the arrows represents the 
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confidence of the detected plane. It can be observed that the man-made surfaces (e.g., pavement 
or sidewalk) show much higher confidence than the natural surfaces(e.g., grass) where the 
terrain may slightly distort the surface within the stripe. After all the stripes are processed, the 
segmentation results are clustered longitudinally based on the k-nearest neighbor (k-NN) 
method (Ai and Tsai, 2016b; Tsai et al., 2013). 


Figure 15 An example of the strip-based sidewalk extraction results 


2.5 Curb Ramp Extraction 

Although curb ramps have unique features, such as shoulder ramps and center ramp patches 
with truncated dome bumps, their diverse appearances and designs make it challenging to 
implement a fully automated curb ramp extraction method. Therefore, this study introduces a 
semi-automatic approach, including an existing automatic curb ramp extraction algorithm and 
a manual curb extraction tool. 


2.5.1 Automated Curb Ramp Extraction 


As suggested in the literature review, the research team directly adopted an existing curb ramp 
extraction algorithm developed by Hara et al. (2013, 2014) and implemented by Ai and Tsai 
(2016b) based on the Deformable Part Model (DPM) (P. Felzenszwalb et al. 2008). The DPM 
algorithm consists of two primary steps: a coarse model (i.e., root filter) and a fine model (i.e., 
parts filter). Originally, the DPM was proposed for human extraction and pose estimation, 
where the root filter captures the overall body outlook from the image, and the parts filter 
captures the individual parts of the human body. In this study, the root filter captures the overall 
appearance of a curb ramp (as shown in Figure 16 in blue) while the parts filter captures the 
individual components of a curb ramp (as shown in Figure 16 in orange). 
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Figure 16 Illustrations of the results from the root filter (blue) and parts filter (orange) 


2.5.2 Manual Curb Ramp Extraction 


In this study, an efficient manual curb extraction tool using Potree (Schutz, 2016) was 
developed to review, update, and edit the automated curb ramp extraction results. Potree is a 
JavaScript-programmed, octree-enabled, web-based library that can efficiently visualize the 
raw point cloud data and then conveniently annotate the point cloud using tools, such as a 
bounding box, polyline contour, etc. so that all the curb ramps can be reliably and efficiently 
detected. Figure 17 shows the interface of the manual curb ramp extraction tool, with a curb 
ramp manually labeled. 


Figure 17 The interface of the manual curb ramp extraction tool 
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2.6 Key Feature Measurement: 

The key feature measurements of interest in this study include sidewalk width, cross slope, 
grade, and curb ramp slope (running slope). In this subsection, the key features associated with 
the sidewalk and curb ramp are discussed, respectively. 


2.6.1 Sidewalk Features 


The key features of interest for the sidewalk, including width, cross slope, and grade, are linear 
features that require continuous measurements. In this study, an interval of 10 ft. was selected 
for sampling the width and cross slope measurement to balance the detail and redundancy of 
the measurements, while an interval of 40 ft. was selected for sampling the grade measurement 
to avoid measuring a local deformation instead of the actual elevation changes along the 
longitudinal direction. Figure 18 illustrates how a single measurement was conducted. 


Sample# = 002547 
~ Grade = 0.77% 


Width = 9.534 ft. 
Cross Slope = 2.60% 


(c) 


Figure 18 Illustrations of the sidewalk measurements for width, cross slope, and grade 


Each blue dot shown in Figure 18 represents an anchoring point with a fixed interval of 10 ft. 
At each anchor point, the width is measured between the two intersection points h1 and h2. 
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Both points are the intersecting points between the boundary of the detected sidewalk and the 
vector that is perpendicular to the data collection trajectory (i.e., the blue line shown in Figure 
18). The x-y distance between h1 and h2 are recorded as the width of the sidewalk at the current 
anchoring point, whereas the linear regression result from all the points between h1 and h2 
(with a buffer of a narrow strip of 3 ft.) is recorded as the cross slope at the current anchoring 
point. Similarly, for every four anchoring points (i.e., every 40 ft.), the grade is recorded based 
on the linear regression results from all the points between h3 and h4 (with a buffer of a narrow 
strip of 1 ft.). 


2.6.2 Curb Ramp Feature 


The key feature of interest for curb ramps, the slope, is defined as the running slope at a curb 
ramp, as shown in Figure 19. However, the detected curb ramp contains more than just the 
running slopes, because the landing slope and flare slope may be included in the extraction 
results, as shown in Figure 19. Therefore, in this study, a similar splitting and merging 
algorithm, as proposed in Section 2.4, is applied to all the detected curb ramp-associated point 
clouds for further segmentation into distinctive sloped sections. Figure 19 shows the three 
examples of the detected curb ramps with different sloped sections. 


Figure 19 Examples for various running slopes of curb ramps 


In Figure 19(a), only one sloped section is identified. Therefore, the running slope is computed 
based on the only deviation angle from the normal direction of the surface to the vertical 
direction. In Figure 19(b) and (c), three or more sloped sections are identified. Therefore, the 
running slope is computed based on the deviation angle from the normal direction of the largest 
surface in the area to the vertical direction. 
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3 Results 


3.1 Results of Acquired Data 

Using the vertical configuration as presented in Section 2.3, the complete State Route 9 
corridor was collected in three separate runs to ensure the entirety and the quality of the data, 
in cases that the data collections were occasionally affected by heavy traffic (i.e., occlusion of 
the point cloud) or poor weather (i.e., heavy fog and rain). Figure 20 shows an example of the 
collected point cloud data with a zoom-in view of the pedestrian facilities. The final dataset 
was prepared by merging different data acquisition sessions for the complete State Route 9, 
which consists of approximately 8 billion LiDAR points and covers all 271.76 miles of the 
corridor. 


Figure 20 An example of the collected mobile LiDAR point cloud 


3.2 Results for Sidewalk Inventory 


The objectives of the sidewalk inventory are threefold: 1) verify the sidewalks located in the 
existing inventory dataset, 2) update sidewalk width, and cross slope at 10 ft. intervals, and 3) 
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understand the feasibility and productivity of the proposed method. The results for the sidewalk 
inventory with respect to those three objectives are as follows. 


3.2.1 Updated Sidewalk Locations 


Complete sidewalk inventory results were obtained using the proposed method from the State 
Route 9 corridor for both bounds, covering more than 85 miles of sidewalks. Figure 21 shows 
several examples of the inventoried sidewalk in urban, suburban, and rural areas, where the 
red polyline layer highlights the detected sidewalk (including the boundaries) using the 
proposed method in this study, while the yellow and green polyline layers highlight the 
inventoried sidewalk in the existing dataset from MassDOT. Several key findings were 
revealed: 


1. Completeness of the dataset: The existing sidewalk inventory records approx. 76.8 miles 
of sidewalk from both bounds of the State Route 9, while the results obtained from the 
proposed method covers approx. 85.1 miles of sidewalk from both bounds. A detailed 
investigation revealed that many sections of the sidewalk were not inventoried. The newly 
inventoried sections are attributed to two primary reasons: 1) there are multiple newly built 
sidewalks that were not updated in the existing database, and 2) there are multiple sections 
of sidewalks on one bound that were not inventoried. While the overall existing sidewalk 
inventory shows a relatively high level of completeness, the inventory results obtained 
from the proposed method shows a significantly higher level of completeness. A complete 
manual review is still required for a full inventory of the sidewalk, but as the proposed 
method can accurately extract 94.3% of the sidewalk, the manual intervention becomes 
negligible 

2. Sidewalk path vs. Sidewalk: It can be observed that the existing sidewalk inventory 
collected sidewalk paths instead of the paved sidewalks. A sidewalk path represents the 
pathway that a pedestrian or a wheelchair user can travel upon, including both the sidewalk 
and the connectors such as crosswalks and driveways. Therefore, the existing sidewalk 
inventory contains connected, continuous sections instead of fragmented sections. On the 
other hand, the proposed method only extracts the sidewalk sections without identifying 
the connectors due to the nature of the LiDAR segmentation and sidewalk extraction 
algorithms. Therefore, the new sidewalk inventory contains many small sections that are 
separated by the connectors. From a planning perspective (taking into consideration 
connectivity, accessibility, etc.), identifying the sidewalk path may serve a better purpose. 
However, from a maintenance perspective, identifying the exact locations of the paved 
sidewalk may serve a better purpose. Therefore, in addition to the updated sidewalk 
inventory, the research team also developed a separate method for identifying the cross 
work and driveway, so that these connectors for completing the sidewalk path can be 
extracted into separate geodatabases. Therefore, by merging or splitting the sidewalk 
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inventory data layer and the sidewalk path inventory data layer, the newly obtained 
geodatabase could provide better support for both planning and maintenance needs. 


Rural Suburban 


Figure 21 Example of the detected sidewalks in the urban, suburban and rural regions 


3. Missing sections: Through a thorough review of the obtained results from the proposed 
method, there are only a few sections that were missed by the algorithms. These missing 
sections are attributed to the excessive occlusion due to densely parked vehicles or heavy 
pedestrian volume. Such an issue can be overcome by multiple data collection sessions 
and better data collection schedule. In addition, it should be noted that the data collection 
in this study was conducted using a passenger vehicle with a sensor installation height of 
5 ft. A higher installation height could avoid many potential obstructions in the future. 


3.2.2 Updated Sidewalk Measures 


With the updated sidewalk locations using the proposed method, the corresponding widths, 
cross slopes, and grades were measured at the intervals of 10 ft. and 40 ft., respectively. Figure 
22 shows an example of the measurements along a small section of 1000 ft. It can be observed 
that in this section, the sidewalk width is consistently around 6.5 ft. with minimum variance, 
while the cross slopes may vary from 0.4% to 0.6%, and the grade was slightly uphill across 


32 


the section varying from 0.2% to 0.5%. Each yellow dot represents a record in the point feature 
class that consists of a set of measurements. 


5 
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Figure 22 An example of the measurements along a 1000-ft. section 


Along the entire State Route 9 corridor, the sidewalk width varies from 3.3 ft. (which requires 
attention for further improvement) to 14 ft. (which provides abundant width for wheelchair 
users and pedestrians), while the sidewalk cross slope varies from 2.5% (requiring attention 
for further safety improvement) to -0.7% (may require drainage improvement). With a 
dedicated georeferenced layer of sidewalk measurements at 10 ft. and 40 ft. intervals, it 
becomes very convenient for MassDOT to identify the sections that may require further 
investigation or improvement. The research team prepared the complete sidewalk inventory 
results in a file geodatabase that consists of one polyline-feature class (sidewalk location), two 
point-feature classes (10 ft. measurements for the width and cross slope, and 40 ft. 
measurements for a grade). 


3.2.3 Feasibility and Productivity 


The updated sidewalk locations and measurements provide MassDOT a convenient sidewalk 
inventory dataset for filtering through sections of interest and sections for improvement. While 
the proposed method shows improved results with location and measurement accuracy, the 
objective of this subsection is to evaluate whether the proposed method is productively feasible 
for practical uses if it is implemented on a larger network. 
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In this study, the entire State Route 9 in Massachusetts, covering all 271.76 miles of the road 
networks, were processed. Along the State Route 9 corridor, more than 85 miles of sidewalk 
locations were identified. Using a conventional work-station (with Intel i7-8700 CPU, 
NVIDIA GeForce GTX 1080 and TensorFlow 1.8), the proposed method created the complete 
sidewalk inventory along the corridor in less than 30 hours, which can be translated into a 
processing speed of approximately 6.5 min/mile (or 0.2 million pts/sec). As mentioned above, 
although a full manual review is still required for a full inventory, the 94.3% accuracy of the 
proposed method affords a minimal effort during the manual review (i.e., minimal digitization) 
of less than 2 hours; that can be translated into the review speed of approximately 0.5 min/mile. 
With parallelization of the program on multiple CPUs and GPUs, it is anticipated that the 
proposed method has the potential to at least achieve a faster processing speed than the current 
data acquisition (i.e., 1.5 min/mile at 40mph). For example, if a Quad-GPU configuration is 
built for processing the point cloud data, it is anticipated that the processing speed would be 
close to 1.7 min/mile. 


Comparing the manual digitization from satellite imagery and field survey, the proposed 
method provides a feasible solution to accurately and efficiently inventory the network-level 
sidewalk information. Moreover, the proposed method has the capacity to obtain the width, 
cross slope, and grade information at a dense interval of 10ft. and 40ft. respectively, which was 
previously not possible. 


3.3 Results for Curb Ramp Inventory 


3.3.1 Updated Curb Ramp Locations 


The current curb ramp inventory effort made by MassDOT includes a geodatabase that 
contains 1,283 locations of curb ramps along State Route 9, as shown in Figure 6. In this study, 
the proposed semi-automated curb ramp extraction algorithm was applied to update the 
locations of the existing inventory (i.e., to add newly identified curb ramps and to remove 
redundant or misclassified curb ramps), and to update the running slope information for the 
validated inventory. 


An addition of 17 curb ramps was newly identified by the proposed method, and 3 inventory 
curb ramps were removed, which formed a complete curb ramp inventory that consists of 1,297 
entries. In addition, the proposed method automatically updated 345 locations of the existing 
inventory with new x-y coordinates that are consistent with the collected LiDAR point cloud 
data. Figure 23 shows an example of the update, where dark blue dots represent the updated 
locations, and light blue dots represent the original locations. It can be observed that the 
location identified by the proposed method is consistent with the ESRI imagery; while the 
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original location was slightly offset, which may be skewed due to the conversion of the satellite 
images and the GPS records from the field survey. 


Figure 23 An example of the updated curb ramp locations 


For measuring the running slopes of the inventoried curb ramps, the proposed method 
automatically measured all 1,297 curb ramps and generated the measurements for running 
slopes, which ranged from 2.7% to 9.3%. Although a further on-field investigation is still 
needed, the locations where running slopes are greater than 8.3% are recommended for further 
attention and inspection. While no field survey was carried out as part of this study due to the 
limitation of the funding and time frame, previous studies by the PI and his research team has 
validated the LiDAR-based method can achieve a running slope accuracy with an error rate of 
0.1% in comparison to the previous study by Ai and Tsai, 2016b. The research team prepared 
the complete curb ramp inventory results in a file geodatabase that consists of one point-feature 
class (curb ramp location and measurements). 


3.3.2 Feasibility and Productivity 


Although the curb ramp inventory and measurement include a semi-automated extraction 
method that requires manual intervention after the automated algorithm completes its 
processing, the overall productivity of the process remains encouraging thanks to the 
convenient tool developed based in Potree. In this study, the entire State Route 9 in 
Massachusetts, covering 271.76 miles of the road networks, was processed. Along the State 
Route 9 corridor, 1,297 curb ramps were identified or updated using the same conventional 
workstation utilized for the sidewalk inventory. Among the 1,297 curb ramps, the fully 
automated method identified 994, while the manual intervention processed the remaining 303. 
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It should be noted that although only less than 23.3% of the curb ramps were missing from the 
automated process, manual intervention is still required to review the entire corridor. Once all 
the curb ramps were identified, the locations for these ramps and the measurements were 
automatically populated into the geodatabase. In this study, the proposed method creates the 
complete curb ramp inventory along the corridor in less than 10 hours; this can be translated 
into the processing speed of approximately 2.2 min/mile. For each mile of data, the algorithm 
processed approximately 320 frames of images and the corresponding point cloud in less than 
45 seconds, while the remainder of the 87 seconds was spent on the manual intervention using 
the interactive tool. Comparing with the reported rate of 30 min/curb ramp (Ai and Tsai 2016c) 
using a field survey method with handheld GPS devices, or 5min/curb ramp (Ai and Tsai 
2016c) using a satellite image digitization method, the proposed method provides a more 
efficient means for network analysis for curb ramp inventory, and more importantly, for 
accurate curb ramp geometrical measurement. 
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4 Conclusion 


This study presents a new pedestrian infrastructure inventory and geometry measurement 
method by leveraging emerging mobile LiDAR, deep learning, and computer vision 
technologies. The method consists of two computer-aided algorithms with measurements for 
sidewalk inventory and curb ramp inventory and was validated on the complete State Route 9 
corridor for its network-level feasibility. 


The proposed method automatically processed State Route 9 in Massachusetts, covering 
271.76 miles of its road networks using a conventional workstation. The results of this study 
show that the proposed method is a feasible means for a large-scale pedestrian infrastructure 
inventory, both in terms of accuracy and efficiency. Two complete _geodatabases for 


sidewalk inventory and measurement and for curb ramp inventory and measurement. 
were generated using the proposed methods. 


The results show that mobile LiDAR is an effective and efficient technology for network-level 
pedestrian infrastructure inventory: 


1. The derived sidewalk inventory geodatabase consists of one polyline-feature class for 
sidewalk locations, and two point-feature classes for width and cross slope measurements 
(10 ft. interval), and grade measurements (40 ft. interval). Along the entire State Route 9 
corridor, more than 85 miles of sidewalk were identified and updated with the existing 
inventory. 

a. The sidewalk width varies from 3.3 ft. that requires attention for further 
widening improvement to 14 ft. that provides abundant width for wheelchair 
users and pedestrians, while the sidewalk cross slope varies from 2.5% that 
requires attention for further safety improvement to -0.7% that may require for 
drainage improvement; 

b. The proposed sidewalk extraction and measurement algorithm automatically 
generated the complete sidewalk inventory along the corridor in less than 30 
hours, followed by a manual review that only requires an additional 2 hours. 

2. The curb ramp inventory geodatabase consists of one point-feature class for curb ramp 
locations and measurements. Along State Route 9, a total 1,297 curb ramps were identified 
and updated with the existing inventory. 

a. Along the entire State Route 9 corridor, the running slope of curb ramps ranges 
from 2.7%, which provides a comfortable riding or walking experience, to 9.3% 
which may require further improvement for landing safety; 

b. The proposed curb ramp extraction and measurement algorithm automatically 
generates the complete curb inventory along the corridor in less than 3 hours, 
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followed by a manual review and update that only requires an additional 7 
hours. 


3. With the clear demonstration of the feasibility of using mobile LiDAR data for pedestrian 


infrastructure inventory, the research team recommends implementing the proposed 
method for a pedestrian infrastructure inventory on a larger network. 


The research team developed a complete point cloud processing pipeline (algorithms, tools, 
and procedures) for pedestrian infrastructure inventory and many other critical transportation 
assets. 
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In this study, the computer-aided algorithms for both sidewalk inventory and curb ramp 
inventory were established upon the existing method developed by Ai and Tsai (2016c). 
The research team further improved the algorithms by addressing the technical challenges 
in this method, including 1) the existing method relies on strong priors, such as sensor 
configuration and scene context, to tune the parameters for consistent results; 2) the method 
can only handle simple scenes where features of interest are significant; 3) the method 
relies on an iterative or sequential search and a tedious matching strategy for feature 
extraction. In particular, the PointNet++-based LiDAR segmentation algorithm proposed 
in this study was not only intended for sidewalk extraction but also to lay a strong 
foundation for a generalized, LiDAR-based, deep-learning-enabled transportation 
infrastructure inventory framework. The general framework for the segmentation can be 
tailored towards facilitating other critical transportation infrastructure and asset extraction 
with minimum changes. 

In addition to the two computer-aided algorithms for sidewalk and curb ramp inventories, 
a manual LiDAR point cloud processing interface was also developed to complement the 
automated curb ramp extraction algorithm, as well as to provide a convenient tool to 
visualize, update and edit the sidewalk and curb ramp inventory information. It should be 
noted that this Potree-based LiDAR point cloud processing interface was not only intended 
for sidewalk and curb ramp measurement or result review, but also to lay a strong 
foundation for facilitating a simple, yet powerful interactive LiDAR point cloud processing 
tool. This would allow it to be easily utilized by MassDOT for visualizing, reviewing, and 
editing the LiDAR point cloud or point cloud-derived information without the hassle of 
configuring complex workstation environment. 

With the establishment of the mobile LiDAR processing pipeline (algorithms, tools, and 
procedures), the research team recommends gaining more insight into using mobile LiDAR 
data to conduct inventory and condition evaluations for other critical transportation 
infrastructure and assets, including pavement marking, signage, guardrail, etc., to increase 
the positive impact the technology will have in subsequent uses. 
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